Analysis of Patient - Robot Interaction Using Statistical and Signal Processing

SWANGNETR, MANIDA. Analysis of Patient-Robot Interaction Using Statistical and Signal Processing Methods. (Under the direction of Drs. David B. Kaber and Yuan-Shin Lee.) Due to a shortage of nurses in the U.S., future healthcare service robots are expected to interact directly with patients. In such a scenario, patient emotional needs may be equally important as traditional performance measures of patient-robot interaction (PRI), including efficiency and accuracy. Consequently, there is a need to design nursing robots with the capability to detect and respond to patient emotional states and to facilitate positive patient experiences in healthcare. The objective of this research was to understand the effect of different features of service robots on user perceptions and emotional responses in a simulated medicine delivery task. The research also developed a new computational algorithm for human emotional state classification to facilitate effective PRI. Physiological responses, such as heart rate (HR), galvanic skin response (GSR) and facial electromyography (EMG), have been found to be clear and valid indicators of valence and arousal. Such physiological signals from patients can be monitored in real-time in hospitals. It is possible that physiological measures can be extracted and emotional states classified during patient interaction with robots. This information can provide a basis for real-time adaptation of robot behaviors to optimize patient emotional experiences in, for example, medicine delivery tasks. A first experiment was conducted at nursing homes using a service robot with different human-like features (face, voice and interactivity) to deliver medicine to elderly residents. Physiological signals, including HR and GSR, and subjective ratings of valence (happyunhappy) and arousal (excited-bored) were collected on participants during experiment trials. A three-stage emotional state classification algorithm was applied to these two types of data. The algorithm involved: (1) physiological feature extraction; (2) statistical-based feature selection; and (3) a machine learning model of emotional states. A proposed de-noising algorithm using wavelet analysis was applied to elucidate relationships among physiological responses and patient emotional states. Analysis of effects of service robot anthropomorphic features on patient emotional states revealed individual humanoid features to promote positive arousal and valence states. Results from this study indicated strong non-linear relationships between the physiological variables and emotional states. Arousal was significantly explained by GSR features while valence was better explained by HR and a small set of GSR features. A second experiment was conducted to assess the effect of combinations of robot physical appearance and interface features and to examine facial EMG signals as another physiological measure for predicting patient emotional states. This experiment was conducted with younger participants (<40 years of age) to map a broad range of service robot users in healthcare environments. Results revealed combinations of humanoid features to increase user arousal and valence. However, multiple robot features led to emotional uncertainty (variability) and intensity for the younger participants, which were largely reflected in physiological responses. Strong non-linear relationships were observed between the physiological variables and emotional states with a similar pattern to the first experiment. EMG activity for cheek and brow region muscles also proved to be an effective valence state indicator. However, smiling tended to be more significant in predicting arousal states than frowning. The new emotional state classification algorithm may serve as an effective method for service robot real-time detection of patient states and behavior adaptation to promote positive patient healthcare experiences. Due to significant individual differences in variability in emotional responses, variations on this algorithm should be adapted for different user groups. Analysis of Patient-Robot Interaction Using Statistical and Signal Processing Methods